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ABSTRACT 

vJ  v  ‘  '■  *-'■ 

This  paper  presents  a  new  Method  for  the  synthesis  of 
textures  on  3-D  surfaces.  To  opr  knowledge,  one  basic  tech¬ 
nique  has  been  presented  up  to  now  in  the  literature  — 

[7-18] )  In  this  standard  method,  textures  are  synthesized  by 
mapping  a  rectangular  template  onto  the  curved  surface.  This 
method  is  complex,  requires  substantial  computing  time,  and 
presents  some  drawbacks  such  as  the  possibility  of  obtaining 
aliasing  effects  and  continuity  problems  along  the  edges  of  the 
curved  templates,  Procedures  to  eliminate  these  problems  are 
available  [11,12]  but  make  this  synthesis  even  more  unattrac¬ 
tive.  The  method  proposed  in  this  paper  does  not  present  the 
former  drawbacks.  We  do  not  use  a  template  mapping,  which  is 
a  drawback  in  itself.  The  synthesis  is  achieved  continuously 
on  the  surface,  so  that  there  are  no  edge  effects  and  also  no 
aliasing  effects.  This  method  is  a  simple  extension  qZ  -a.  pro¬ 
cedure  that  we  have  proposed  before  in  the  literature  [2,3]  for 
planar  textures.  Any  kind  of  texture  can  be  reproduced  with 
a  good  similarity  to  the  reference  texture  used.  It  also  has 
the  important  advantage  that  only  one  set  of  second  order  sta¬ 
tistics  (a  small  amount  of  data)  needs  to  be  computed  on  a  pla¬ 
nar  version  of  the  reference  texture  to  synthesize  this  texture 
on  any  surface  and  at  any  distance.  Some  results  on  simple  sur¬ 
faces  are  displayed  (cylinder,  sphere) ,  but  the  method  holds 
for  any  surface  and  is  relatively  quick  and  easy. 
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1 .  Introduction 


The  generation  of  images  by  computer  is  becoming  more 
and  more  popular  nowadays.  It  has  been  applied  industrially 
mainly  for  flight  simulators  in  the  past  few  years,  but  it  is 
becoming  more  and  more  utilized  for  entertainment.  A  number 
of  movie  films  generated  by  computer  have  already  been  pre¬ 
sented  on  television,  and  even  in  theaters  in  the  case  of  the 
movie  TRON. 

The  techniques  available  in  computer  graphics  are  already 
very  powerful  for  the  representation  of  3-D  scenes  including 
various  illumination  effects  (shadows,  transparency,  varia¬ 
tions  of  the  reflectance) ,  but  their  performance  is  still  poor 
in  the  case  of  textured  surfaces.  The  purpose  of  this  work 
is  to  find  a  solution  to  this  problem.  We  think  that  the  tex¬ 
ture  synthesis  procedure  described  in  this  paper  should  allow 
the  generation  of  much  more  realistic  images  (since  we  are 
able  to  reproduce  any  kind  of  texture  with  good  visual  similarity) 
using  a  rather  small  amount  of  data.  Only  one  set  of  second 
order  statistics  per  texture  is  sufficient  to  synthesize  it  on 
any  surface  having  any  extent.  This  method  is  a  generalization 
of  the  synthesis  procedure  we  have  designed  for  planar  textures 
[2,3]  to  the  case  of  3-D  surfaces. 

Given  a  reference  feature  vector  constructed  by  the  con¬ 
catenation  of  the  second  order  statistics  (second  order  spatial 
averages  or  autocovariance  +  histogram)  of  a  planar  texture 
field,  we  construct  the  projected  image  of  a  3-D  textured  surface 
while  minimizing  the  mean  square  error  between  the  feature  vector 


of  the  current  3-D  texture  and  the  reference  feature  vector. 

The  synthesis  is  achieved  point  by  point,  sequentially,  thus 
yielding  good  continuity  properties  on  the  surface  so  that  we 
have  neither  aliasing  effects  nor  discontinuity  effects,  which 
usually  occur  with  the  standard  procedure  available  in  the 
literature  [7-13] .  The  drawbacks  of  the  standard  method  are 
inherent  to  its  nature  because  the  synthesis  is  done  by  mapping 
a  rectangular  planar  patch  of  texture  onto  the  curved  surface. 
The  techniques  used  to  eliminate  aliasing  effects  and  conti¬ 
nuity  problems  along  the  boundaries  of  the  patches  [11,12] 
make  the  method  difficult  to  use  in  practice.  All  these  rea¬ 
sons  favor  the  new  synthesis  method  proposed  here. 


Definitions  and  notation 


Using  standard  computer  graphics  terminology,  we  define 
object  space  as  the  three-dimensional  space  in  which  an  object 
will  ordinarily  be  described.  In  order  to  generate  realistic 
images  of  objects,  we  make  a  perspective  transformation  of 
the  object  from  the  object  space  into  the  image  space.  The 
image  space  is  also  three-dimensional,  but  objects  have  under¬ 
gone  a  perspective  distortion.  An  orthogonal  projection  of  the 
image  onto,  let  us  say,  the  x-y  plane  results  in  the  expected 
perspective  image,  which  will  be  called  a  projected  image.  Sine 
the  main  purpose  of  this  work  is  texture  synthesis,  for  simpli¬ 
city  we  shall  suppose  that  the  viewpoint  is  at  infinity  so  that 
object  space  and  image  space  will  be  identical  (and,  from  now 
on,  will  be  called  object  space) ,  and  a  projected  image  will  be 
called  simply  an  image. 

A  texture  is  considered  to  be  a  realization  of  a  stochastic 
process  x  defined  on  a  domain  S.  S  is  a  digital  surface  in  a 
3-D  space,  x  takes  its  values  on  a  finite  set  L  of  grey  levels: 

L  =  {0,1, .. . , L-l } 

All  points  of  S  can  be  indexed  by  their  rank  i  (i=l,2,...,N 
where  N  is  the  number  of  points  in  S) ;  x^  will  represent  the 
value  of  x  at  location  i.  We  shall  deal  with  homogeneous  and 
ergodic  processes,  at  least  up  to  the  second  order,  so  that  pro¬ 
babilistic  parameters  and  estimates  of  these  parameters  will  be 
identical.  Thus,  first  order  probability  distributions  p(£)  can 
be  confused  with  histogram  parameters  p(&)  and  autocovariance 


moments  M2 (A) /  where 


p(£)  =  p(xm=£)  Vm, & 

P(“)  =  |  Z  ” 

m=l 


(1) 


M2  (A) 
M2  (A) 


E[<xm-r,>*<Xm+4-il)]  VA 

7T  \  (xm-’l)‘(xm+4-T') 

la  m=l 
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and 

•n  -  5  \  xn  (-Elxm]> 

m=l 

is  the  texture  mean, 

=  5  \  (xnfr"2  (-El‘xm-E<V>2I> 

m=l 

is  the  texture  variance,  E(.]  denotes  the  mathematical  expecta¬ 
tion;  <5,  the  Kronecker  function;  A,  a  translation  in  the  3-D 
space;  and  I  is  the  number  of  pairs  m,  m+A  belonging  to  S.  A 
more  extensive  discussion  of  these  homogeneity  and  ergodicity 
properties  of  textures  can  be  found  in  [1,4]. 

Psychovisual  experiments  [1,4]  have  shown  that  planar  tex¬ 
tures  are  well  modelled  by  second  order  spatial  averages 
Pa(L1,L2)  where  A  varies  in  a  2-D  space  such  that  ||A||s9'  of 
solid  angle  (approximately) .  We  have  also  shown  that  a  simpler 
model  with  histogram  parameters  and  autocovariance  parameters 
M2(a),  where  A  varies  in  the  same  range  as  above,  gives  accept¬ 
able  synthesis  results  for  certain  natural  textures.  The  syn¬ 
thesis  procedure  designed  in  this  paper  is  based  upon  the  latter 
model,  but  this  is  by  no  means  a  limitation;  we  could  use  a  similar 


procedure  controlling  second  order  spatial  averages  as  well. 

The  synthesized  textures  would  be  of  a  better  visual  quality, 
but  the  algorithm  would  be  somewhat  more  complicated  and  would 
require  much  longer  computation  time.  An  interesting  property 
of  the  model  we  proposed  for  textures  [1,2,4]  is  that  it  can 
be  extended  immediately  to  the  case  of  textures  lying  on  3-D 
surfaces.  The  fact  that  this  extension  is  easy  reinforces 
the  former  results.  The  general  conjecture  becomes: 

"We  cannot  discriminate  two  textures  if  locally  they  possess 
the  same  second  order  statistics  for  ||a||s9'  of  solid  angle" 

(A  belongs  to  a  3-D  space  in  the  case  of  a  3-D  texture) . 

In  practice,  the  problem  we  have  to  solve  is  to  synthesize 
a  realization  of  a  homogeneous  stochastic  process  defined  on 
a  3-D  surface  from  a  priori  given  second  order  statistics  (in  our 
case,  histogram  +  autocovariance  function)  for  i|  A i| < 9 •  approxi¬ 
mately.  Locally,  a  texture  can  be  considered  as  a  distorted 
version  of  a  planar  texture  so  that  we  can  suppose  that  our 
input  data  were  computed  on  a  plane.  A  then  belongs  to  a  2-D 
space  and  our  model  of  texture  becomes  a  planar  model,  which 
is  easier  to  work  with.  But  the  algorithm  will  have  to  take 
care  of  the  geometric  distortions  on  the  3-D  surface.  The  im¬ 
portance  of  the  first  remark  is  that  one  set  of  second  order 
statistics  (computed  on  a  planar  texture  field)  will  allow  us 
to  synthesize  this  texture  on  any  surface.  The  model  is  an 
intrinsic  one;  it  does  not  depend  on  the  type  of  surface  on  which 
the  texture  lies.  This  corresponds  well  to  our  intuitive  under¬ 
standing  of  textures. 


Based  upon  earlier  experiments  [1,4],  the  set  of  transla¬ 
tions  A  we  should  consider  should  be  defined  within  a  circle 
(since  we  are  dealing  with  a  planar  model).  For  simplicity, 
we  decided  to  use  a  set  of  translations  defined  by  a  rec¬ 
tangle  TR'  of  (2Nx+l) * (2Ny+l)  pixels  (see  Figure  1).  This 
rectangle  will  contain  the  previously  considered  circle. 

Since  we  are  dealing  with  a  discrete  array,  all  translations 
A  are  defined  by  a  pair  of  discrete  values  (nx,ny) .  Let 
us  define  TR'  by 

TR'  =  A,  A=(n,m)  Vn  =  -Nx, -Nx+1 , . . . +Nx,  A^(0,0) 

Vm  =  -Ny,-Ny+1, . . .+Ny 
Since  the  autocovariance  function  is  symmetric  (M2 (A)=M2 (-A) ) , 
we  can  restrict  ourselves  to  half  of  the  previous  domain: 

TR  =  |  A,  A=(n,m),  (  (Vn  €  [-Nx,Nx]  )  fl  (m€  [1  ,Ny]  )  U 

<(Vn  ^  [-Nx,-l]n  (m=0) )} 

The  total  number  of  translations  belonging  to  the  set  TR  is 
then  equal  to 

2  *Nx*Ny+Nx+Ny . 

For  example,  in  the  case  where  Nx=Nx=l,  this  number  is  equal 
to  4 . 

We  denote  by  B  the  vector  containing  the  autocovariance  para 
meters  related  to  the  various  translations  A  belonging  to  TR, 
and  by  H  the  vector  containing  the  histogram  coefficients. 


4 .  Synthesis  procedure 
4. 1  Principle 

The  synthesis  is  achieved  directly  on  the  object  surface, 
but  since  there  is  a  one  to  one  correspondence  between  the 
object  and  its  projected  image,  we  need  only  to  manipulate  the 
points  of  the  image  plane  (see  Figure  3).  Suppose,  for  in¬ 
stance,  that  the  viewpoint  is  at  infinity  along  the  positive  z 
axis  as  shown  on  Figure  3.  Let  us  denote  by  S'  the  projection 
of  S  on  the  z=0  plane.  S'  is  the  visible  part  of  S  from  our 
viewpoint.  If  i  is  a  location  on  the  surface  S,  let  us  denote 
by  i'  its  projection  on  the  image  S'  of  S.  For  simplicity, 
we  consider  that  there  is  no  shadow  effect  and  that  the  reflec¬ 
tance  is  constant,  though  such  effects  could  also  be  treated  by 
our  method.  If  x^  is  the  luminance  of  the  texture  at  location 
i,  the  luminance  of  the  image  at  location  i'  is  then  x|,  where 
x'  can  be  considered  as  a  stochastic  process  defined  on  S ' . 

With  this  hypothesis,  we  have 

xi'  =xi 

so  that  it  is  equivalent  to  manipulate  points  of  S'  or  points  of 

5.  We  suppose,  of  course,  that  the  3-D  surface  S  is  known,  so 
that  we  know  its  visible  part  and  the  correspondence  i  Z  i'  is 
one  to  one.  If  the  surface  is  known,  we  can  determine  the  tan¬ 


gent  plane  to  the  surface  S  at  each  location  i.  We  suppose 
that  we  also  know  the  local  coordinate  system  (u^,v^)  associated 
with  Pi  for  each  location.  This  information  is  struc tural  infor 
mation  telling  us  how  the  texture  is  positioned  locally  on  the 
surface  S.  This  type  of  information  is  a  free  parameter  and  is 
an  input  to  our  synthesis  procedure.  Of  course,  this  informa¬ 
tion  is  constrained  by  the  fact  that  the  local  coordinate  system 
has  to  vary  smoothly  on  the  surface  as  a  function  of  i. 


The  only  restrictive  hypothesis  that  we  are  going  to  make 
is  that  if  we  consider  the  neighborhood  TR'  of  location  i  on 
the  tangent  plane  then  all  points  of  TR'  also  lie  on  the 
surface  S.  This  means  that  the  surface  S  is  smooth  enough 
to  be  approximated  by  its  tangent  plane  up  to  a  length  of 
max(Nx,Ny)  around  each  location  i.  This  hypothesis  implies  strong 
differentiability  constraints  on  the  surface  which  may  not  al¬ 
ways  be  true. 

Given  this  hypothesis,  we  observe  from  (1)  and  (2)  that  the 

feature  vector  related  to  the  surface  S  corresponds  precisely 

to  the  set  TR  of  translations.  Now  it  is  possible  to  solve 

our  problem.  We  have  to  construct  a  texture  field  Tx  on  S  such 

Tx 

that  its  feature  vector  B  will  be  equal  to  an  a  priori  given 
vector  B.  The  synthesis  is  achieved  in  two  steps  in  a  very 
similar  way  to  that  used  in  our  earlier  publications  [2,3]. 
First,  we  create  a  texture  field  on  S'  (or  S,  equivalently) 
which  is  a  realization  of  homogeneous  white  noise  such  that 
its  histogram  H  '  is  equal  to  the  desired  histogram  H.  Then, 
at  the  second  step,  the  texture  is  modified  point  by  point, 
sequentially,  in  a  homogeneous  way,  while  minimizing  a  criterion 
which  is  basically  the  mean  square  error  between  the  desired 
feature  vector  and  the  current  one. 


There  are  three  different  inputs  to  the  synthesis  procedure 
•  First,  the  feature  vectors  H  and  B  containing  the  his¬ 


togram  and  the  autocovariance  parameters  related  to  the 
translation  set  TR  of  the  desired  texture. 

•  Second,  the  x,y,z  sampled  coordinates  of  the  surface  S 

•  Third,  the  local  coordinate  system  (u^,v^)  on  the  tan¬ 
gent  plane  of  the  surface  at  each  location  i  (see 
Figure  3)  . 

This  set  of  inputs  shows  that  there  are  many  degrees  of  free¬ 
dom  in  the  construction  of  a  texture  on  a  3-D  surface. 


1 


4 . 3  First  step  of  the  synthesis  procedure 

Given  a  histogram  vector  H,  we  compute  its  cumulative 

function  (integral).  Using  a  random  number  generator,  we 

can  immediately  synthesize  white  noise  Tx  such  that  its  his- 
Tx 

togram  H  is  equal  to  H.  Tx  is  defined  on  S'.  After  this 

Tx 

step,  the  feature  vector  B  of  the  synthesized  texture  is 
equal  to  zero  since  Tx  is  white  noise  (M2 (A)  =  0,VA).  If 
B=0,  the  procedure  is  stopped  at  this  point.  Otherwise, 
we  have  to  proceed  to  the  second  step. 


s 


4. 4  Second  step 

We  modify  the  texture  field  Tx  defined  on  S'  sequentially 
while  minimizing  the  criterion 
Err  =  ||B-BTx||  2  +  a  ||H-HTx!|2 

where  a  is  a  large  number.  This  criterion  is  roughly  equiva- 

Tx  2 

lent  to  the  minimization  of  IIB-B  under  the  constraint 


Suppose  that  we  are  at  location  i,  that  x^=L.,  and  that 

Tx  Tx  Tx 

B  and  H  w^e  known  {they  are  initialized  by  B  =0  and 

Tx 

H  =H  after  the  first  step) .  Suppose  that  we  replace  by 

Tx  Tx 

•  The  idea  is  to  determine  the  new  values  of  B  and  H 
We  have  to  consider  the  various  neighbors  of  x^  on  the  sur¬ 
face  S  (or  equivalently,  on  the  tangent  plane  Pi) .  The  updat- 
Tx 

ing  of  H  is  given  by: 

~Tx  .  .  -Tx,r  .  1 

p  (L1)  -  p 

~Tx  .  , .  _  ~Tx,r  *  .1  (3) 

P  (L^)  P 

~  Tx 

For  a  given  translation  A  belonging  to  the  set  TR,  M2  (A)  is 
updated  by 

M2TX(A)  -  M2TX(A)  +  (xi+A+Xi_A-2r1)  ,  (4) 

la 

We  have  neglected  the  contributions  of  the  variations  of  r)  and 
a  in  the  updating  of  M2(A)  since  they  are  second  order  terms. 
Usually  (i+A)  and  (i-A)  are  not  sample  points  of  S;  in  other 
words,  their  projections  (i-A) '  and  (i+A) '  on  the  image  S'  are 
not  sample  points  (see  Figure  4) .  x^  ^  and  are  computed 

by  bilinear  approximation  taken  from  the  four  nearest  neigh¬ 
bors  of  (i+A)'  and  (i-A)'  on  S'. 
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If  we  denote  by  Err(L^)  the  value  of  the  criterion  Err 
before  the  updating,  when  x^=L^,  it  is  then  possible  to  compute 
the  new  value  of  the  error  Err(Lp  ,  if  we  replace  L1  by 

Li.  We  simply  apply  (3)  and  (4)  to  obtain  the  new  values 

Tx  Tx  * 

of  H  and  B  .  Finally,  we  replace  by  where 

Err(Ll)  -  Err(Li)  VLl  6  L*  (5) 

As  in  our  earlier  publications  [2,3]  it  is  possible  to  update 

the  error  instead  of  recomputing  the  error  at  each  time,  using 

Err  (Li  )  =  E(L.  )  +  2  (A)  -M,  (A)  )  *  (L.  -L'  ) 

r  1  A€TR  la  ^  1 

*  ,Xi«+Xi-4-2’’  > +  iH  (L1-Li>  2*  (Xi+4+Xi-4‘2T1 1  2 

I  a 

2  ~l  ~  «<ty  ~Ty  ~ 

-  (p  (L- )  +piX  (L1)  -piX  (L*  )  -p  (L1)  )  ] 

All  points  of  S,  or  equivalently  all  points  of  S’ ,  are  scanned 
using  a  random  sequence  so  that  the  criterion  decreases  homo¬ 
geneously  all  over  the  surface  S  (or  S').  (See  [2,3]  for  a 
discussion  of  this  phenomenon.)  The  procedure  is  such  that 
the  criterion  decreases  monotonically  (from  (5))  when  we  up¬ 
date  the  various  points  of  S'. 


5.  Results 


The  first  experiment  was  performed  to  verify  that  our 
synthesis  procedure  gives  correct  results.  For  these  pur¬ 
poses  we  chose  to  achieve  the  synthesis  on  a  cylinder,  which 
is  a  ruled  (developable)  surface,  so  that  it  is  possible  to 
compare  these  results  with  a  synthesis  achieved  starting  from 
a  texture  synthesized  on  a  plane  (which  gives  correct  results 
[2,3])  and  then  applied  to  the  cylinder. 

The  textures  PAPER,  WOOL  and  RATTAN  taken  from  Brodatz 
[6]  were  used.  Figure  5  shows  natural  PAPER  texture  and 
Figure  6  shows  the  result  of  the  synthesis  of  PAPER  using  the 
planar  technique  [3].  Comparison  of  the  two  figures  shows  that 
the  artificial  texture  is  very  similar  visually  to  the  original 
one.  Figure  7  shows  the  natural  PAPER  texture  on  a  cylinder 
seen  at  about  30°  above  the  x-axis  on  the  (x,z)  plane.  We 
simply  applied  the  texture  of  Figure  5  onto  the  cylinder. 

Figure  8  is  a  similar  construction  taken  from  Figure  6.  Figure  9 
displays  the  result  of  our  synthesis  procedure  achieved  directly 
on  the  cylinder  surface.  The  input  feature  vector  consisted 
of  8  histogram  parameters  and  32  autocovariance  parameters 
(NX=NY=2).  The  local  coordinate  systems  (u^,v^)  on  the  surface 
of  the  cylinder  were  chosen  to  be  consistent  with  the  way  we 
applied  the  texture  plane  onto  the  cylinder  in  Figure  7.  The 
u^  vector  was  tangent  to  the  circle  parallel  to  the  base  of  the 
cylinder  and  passing  through  location  i,  and  v^  was  parallel 
to  the  z  axis,  as  shown  in  Figure  10.  Inspection  of  Figures  7, 

8  and  9  shows  that  our  synthesis  procedure  is  correct,  so  that 

the  hypothesis  relating  the  points  of  P,-  to  the  points  of  S  is 
correct  also. 
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Figures  11  and  12  present  a  comparison  of  natural  RATTAN 
applied  onto  the  cylinder  from  the  plane  and  the  direct  syn¬ 
thesis  of  RATTAN  using  NX=NY=12 .  Figures  13  and  14  show 
similar  results  for  WOOL. 

Figures  15  and  16  show  the  synthesis  of  PAPER  and  RATTAN 
textures  on  the  sphere  using  the  same  input  data  B  as  before. 

The  sphere  is  seen  from  the  positive  z  axis  (the  visible  pole 
of  the  sphere  is  at  the  center  of  the  circle).  On  Figures 
17  and  18,  the  sphere  is  seen  from  the  x  positive  axis  (the 
poles  are  not  seen) .  The  local  coordinate  systems  chosen  on 
the  sphere  are  such  that  u^  is  tangent  to  the  parallel  passing 
through  location  i,  and  v^  is  tangent  to  the  longitude  circle 
passing  through  i,  as  shown  in  Figure  19.  The  visual  aspect 
of  the  3-D  textures  is  better  when  the  poles  are  not  seen,  as 
expected.  Another  set  of  local  coordinate  axes  would  give 
another  visual  aspect  of  the  sphere  which  would  correspond  to 
another  way  to  structure  (or  deform)  the  texture  onto  the  sphere 
or  the  cylinder. 

The  reader  can  verify  that  it  is  easy  to  recognize  the 
various  textures  (PAPER,  WOOL  or  RATTAN)  on  the  various  sur¬ 
faces,  and  that  we  do  not  have  any  aliasing  effects  or  any  dis¬ 
continuities  on  the  images.  The  same  40  parameters  are  used 
to  synthesize  PAPER  on  the  plane,  the  sphere  or  the  cylinder, 
and  it  would  be  the  same  for  any  surface.  For  RATTAN  and  WOOL 
we  used  320  coefficients.  The  visual  appearances  of  the  syn¬ 
thesized  WOOL  and  RATTAN  are  not  very  good.  It  would  be  possible 


to  design  a  similar  technique  controlling  the  second  order 
spatial  averages  instead  of  the  autocovariance  parameters. 
This  would  increase  the  visual  quality  of  the  synthesized 
textures.  In  any  case,  the  synthesis  procedure  is  easy  to 
implement  and  we  think  that  such  a  technique  will  have  a 
number  of  applications  in  the  near  future. 
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